Skip to content

Conversation

@jbfeldman-dd
Copy link
Contributor

What does this PR do?

Creates an OCSF pipeline for Nginx logs using the new OCSF processor

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

@github-actions
Copy link

github-actions bot commented Nov 17, 2025

⚠️ Recommendation: Add qa/skip-qa label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

preserveSource: true
overrideOnConflict: true
- type: schema-remapper
name: Map `http.status_code` to `ocsf.status_code`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add string transformation
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should happen automatically without explicit mapping, I escalated to the OCSF team https://dd.slack.com/archives/C07FNKFD7RS/p1763582334628809

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Temporarily explicitly mapped it using targetFormat, should be resolved now. In the future this will automatically

@jbfeldman-dd
Copy link
Contributor Author

OCSF validation output
The two errors can be safely ignored

  • The type_uid issue is due to it being auto-generated in a post-pipeline processor, which the Ci/CD can't account for. Tested and confirmed working in staging
  • The duration issue is due to nginx_test.yaml storing the duration in scientific notation, which the validation script incorrectly reads as a string. Tested and confirmed working in staging
INFO:root:Summary of validation results: {
  "total_logs": 5,
  "total_errors": 6,
  "total_warnings": 5,
  "attribute_required_missing": 5,
  "version_earlier": 5,
  "attribute_wrong_type": 1
}
INFO:root:Error messages: {
  "Required attribute \"type_uid\" is missing.": 5,
  "Attribute \"duration\" value has wrong type; expected long_t, got string_t.": 1
}
INFO:root:Warning messages: {
  "Event version \"1.5.0\" at \"metadata.version\" is earlier than schema version \"1.7.0-dev\". Validating against later schema versions can yield deprecation warnings and other (minor) validation messages that would not occur when validating against the same version.": 5
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants